Investigating different schools of Philosophy–Capitalism vs. Communism

Do you know about Philosophy? If you read a lot about them, that’s very cool. Hope you find this article helpful from the perspective of data analysis and visualization. If you are not very familiar with this subject (just as I am), don’t worry. You will get the first impression on each main school of Philosophy.

The first part of the data story is about analysis of the whole corpus of Philosophy. It will bring you an overview of the data set we are currently working on. The second part focuses on sentiment analysis of two interesting schools: Capitalism and Communism.

Part 1: Data set overview. Let’s look at the whole curpus!

Preparation

Setting up environment

This report is prepared with the following environmental settings.

print(R.version)

               _                           
platform       x86_64-apple-darwin17.0     
arch           x86_64                      
os             darwin17.0                  
system         x86_64, darwin17.0          
status                                     
major          4                           
minor          0.3                         
year           2020                        
month          10                          
day            10                          
svn rev        79318                       
language       R                           
version.string R version 4.0.3 (2020-10-10)
nickname       Bunny-Wunnies Freak Out     

Load the packages and source files

We don’t need all the packages listed there. They covers the functionality from generating, cleaning the original data set, to data analysis, manipulation, visualization, and sentiment analysis etc.. We mainly focus on the latter part of the functions.

I wrote two functions f.clean.corpus and f.make.tdm in the ../lib/wordcloudFuncs.R since we would generate many word clouds using these wrapped functions rather than writing similar, redundant code.

packages.used=c("rvest", "tibble", 
                "sentimentr", "gplots", "dplyr",
                "tm", "syuzhet", "factoextra", 
                "beeswarm", "scales", "RColorBrewer",
                "RANN", "topicmodels", "stringr", 
                "ggridges", "wordcloud", "wordcloud2",
                "tidytext", "knitr", "tidyverse")
# check packages that need to be installed.
packages.needed=setdiff(packages.used, 
                        intersect(installed.packages()[,1], 
                                  packages.used))
# install additional packages
if(length(packages.needed)>0){
  install.packages(packages.needed, dependencies = TRUE)
}
# load packages
library("rvest")
library("tibble")
library("syuzhet")
library("sentimentr")
library("gplots")
library("dplyr")
library("tm")
library("syuzhet")
library("factoextra")
library("beeswarm")
library("scales")
library("RColorBrewer")
library("RANN")
library("tm")
library("topicmodels")
library("stringr")
library("ggplot2")
#library("ggridges")
#library("viridis")
library("wordcloud")
library("wordcloud2")
library("tidytext")
library("knitr")
library("tidyverse")

#source("../lib/plotstacked.R")
source("../lib/speechFuncs.R")
source("../lib/wordcloudFuncs.R")

Load the dataset

The data set ../data/philosophy_data.csv contains 360808 sentences taken from 51 texts spanning 13 major schools of the history of philosophy. You can find more details about it in https://www.kaggle.com/kouroshalizadeh/history-of-philosophy.

file<-'../data/philosophy_data.csv'
full_data <- read.csv(file)
full_data <- full_data %>% 
  mutate(word.count = f.word_count(sentence_spacy))
# sentence.list is generated from the chunk {r generate sentence.list}. This will 
# cost 25-30 minutes. To save your time, you can simply use it.
# Feel free if you don't want to run {r generate sentence.list}
subfile <- '../output/sentence.list.csv'
sentence.list <- read.csv(subfile)

reduced_data = full_data %>% select(title, author, school, sentence_spacy, original_publication_date, sentence_length, word.count)

#trial_data <- reduced_data[1:1000,]
data <- reduced_data %>% filter(school == "capitalism" | school == "communism")

Task 1: Investigate the length of sentences of different schools

The word “philosophy” literally means the “love” (Philo in Greek) of “wisdom” (Sophia). As an outsider, I always think philosophy is a profound, opaque subject which is very challenging to learn. I believe that many people like me would presuppose that sentences in philosophy texts tend to be longer than usual sentences. Hence, let’s begin by investigating the the length of sentences of different schools.

From this bar plot, we see that empiricism, capitalism, and german_idealism are the three schools with longest sentences(longer than 30 words); Nietzsche, analytic, plato are the three schools with relative short sentences(a big longer than 20 words). “In general, an average of 15 to 20 words is effective for most technical communication.” –By James Scott. Hence, we see that even the school with shortest mean length of sentences still has mean length longer than usual written sentences’ length.

data.byschool <- reduced_data %>% group_by(school) %>%
  summarise(meanlength = mean(word.count)) %>%
  arrange(meanlength)
  
g1 <- data.byschool %>% ggplot(aes(x = reorder(school, meanlength), y = meanlength, fill = school)) + 
  geom_bar(stat = "identity") +
  labs(
    title = "Bar plot: mean length of sentences(word count) of different schools",
    x = "Schools",
    y = "Mean Length of Sentences"
  ) +
  coord_flip()

g1

In addition to the mean length of sentences, this violin + box plot will show more details of actual distributions of sentence length. All the schools have outliers(black dots) on the right side of violin plot, which means they all have some very long sentences which might be abstruse or rigorous.

data.byschool <- reduced_data %>% group_by(school) 
g2 <- data.byschool %>% ggplot(aes(x = school, y = word.count, fill = school)) +
  geom_violin(alpha = 0.5) +
  #geom_point(position = position_jitter(seed = 1, width = 0.2)) +
  theme(legend.position = "none") +
  geom_boxplot(width=.1) +
  labs(
    title = "violin plot + box plot: spread of sentences' length \n of different schools",
    x = "Schools",
    y = "Length of Sentences"
  ) +
  coord_flip()

g2

Task 2: Investigate the frequently mentioned words

What we did in task 1 would generally represents the shape and format of sentences in different schools. Now, let’s dig into the contents and topics.

Let’s first have an overview of frequency of words in the whole corpus. A wordcloud would be an optimal method for word frequency visualization.

Prepare corpus

Put all the sentences(360808 observations) into a Large SimpleCorpus docs.

Clean corpus and make document-term-matrices

f.clean.corpus cleans the docs by lowering sentences, removing numbers, punctuations, white spaces, common stop words in English, and an additional set of words rm_words which contains words of my choice. f.make.tdm makes the docs ready to make a wordcloud. See details in ../lib/wordcloudFuncs.R.

rm_words = c('also', 'areas', 'can', 'etc', 'get', 'just', 'like',
'lot', 'many', 'may', 'need', 'one', 's', 'set', 't',
'time', 'us', 'use', 'way', 'well', 'will', 'b', 'e',
'g', 'less', 'give', 'tell', 'im', 'take', 'coming',
'say', 'really', 'must')

docs <- f.clean.corpus(docs)
tdm.overall <- f.make.tdm(docs)

Make a wordcloud of the whole corpus

wordcloud(tdm.overall$term, tdm.overall$`sum(count)`,
          scale=c(5,0.8),
          max.words=200,
          min.freq=1,
          random.order=FALSE,
          rot.per=0.3,
          random.color=FALSE,
          colors=brewer.pal(8,"Dark2"))

Now let’s see the word clouds of different schools.

The method is similar to above. You can read the actual code in ../doc/Philosophy data story.Rmd.

Capitalism:

Empiricism:

German_idealism:

Continental:

Rationalism:

Aristotle:

Feminism:

Communism:

Phenomenology:

stoicism: Thou is an old-fashioned, poetic, or religious word for ‘you’ when you are talking to only one person. It is used as the subject of a verb. In rm_words, I include “you” as it is an insignificant word. But “thou” brings a specific context and style of a philosophy text.

Analytic:

Nietzsche:

Plato:

From these wordclouds, you can find out what words are frequently mentioned in each school of philosophy. And you can imagine what topics they focus on. For my personal interest, I especially analyze the wordclouds of capitalism and communism. Many of us know they are contrast to some degree, but cannot state the differences clearly. I originally expected their wordclouds would be very different. However, both of them mentioned a lot about “value”, “labour”, “capital”, “commodities”, “price”, “wages”, “trade” etc.. The degree of similarity exceeded my expectation, and I began to speculate that they might often talk about similar topic but with contrast ideas. Here, we can refer to Adam Smith, the renowned Scottish Economist(on the side of capitalism), and Karl Marx, a famous Philosopher, and Sociologist, from Germany(on the side of communism), to see the difference of capitalism and communism:

“Capitalism is an economic system in which the trade and industry of the economy is owned and controlled by private individuals, to make profit.”

“Communism refers to social system in which country’s trade and industry are controlled by the community and the share of each individual relies on his ability and needs.”

For more detail, please refer to Difference Between Capitalism and Communism In part 2, we will do some sentiment analysis on these two schools of philosophy, and try to explore something interesting and uncovered by the above article.

Part 2: Sentiment analysis on capitalism and communism

During this part, we apply sentiment analysis using NRC Word-Emotion Association Lexicon. For each sentence, this method would give a score for each sentiment(“anger”, “anticipation”, “disgust”, “fear”, “joy”, “sadness”, “surprise”, “trust”, “negative”, “positive”) according to The NRC Emotion Lexicon, a list of English words and their associations with basic emotions.

task 3 : Investigate and compare emotions between capitalism and communism

generate sentence.list

sentence.list is a subset of the whole corpus. It only includes sentences with school of capitalism or communism. And we will generat their sentiment score and append them on the right of the data set(12 columns: 10 columns of sentiment, 1 column of sentence id, 1 column of word count). It will cost 25-30 minutes to generate sentence.list, so I commented this r chunk and you can use it from “../lib/wordcloudFuncs.R”.

# sentence.list=NULL
# 
# for(i in 1:nrow(data)){
#   sentences <- data$sentence_spacy[i]
#   if(length(sentences)>0){
#     emotions=get_nrc_sentiment(sentences)
#     #word.count=word_count(sentences)
#     # colnames(emotions)=paste0("emo.", colnames(emotions))
#     # in case the word counts are zeros?
#     #emotions=diag(1/(trial_data$sentence_length[1]+0.01))%*%as.matrix(emotions)
#     emotions = as.matrix(emotions)
#     sentence.list=rbind(sentence.list,
#                         cbind(data[i,],
#                               sentences=as.character(sentences),
#                               #word.count,
#                               emotions,
#                               sent.id=i
#                               )
#     )
#   }
# }
#
# sentence.list <- sentence.list %>% left_join(data) #join the word.count into sentence.list
#write.csv(sentence.list,"/Users/jiuruwang/Documents/GitHub/spring-2022-prj1-jrwang0810/output/sentence.list.csv")

names(sentence.list)

 [1] "X.1"                       "X"                         "title"                    
 [4] "author"                    "school"                    "sentence_spacy"           
 [7] "original_publication_date" "sentence_length"           "sentences"                
[10] "anger"                     "anticipation"              "disgust"                  
[13] "fear"                      "joy"                       "sadness"                  
[16] "surprise"                  "trust"                     "negative"                 
[19] "positive"                  "sent.id"                   "word.count"               

Find out emotionally charged sentences

Here we can see some typical short sentences with strong sentiment.

Capitalism:

emotions.types=c("anticipation", "joy", "surprise", "trust",
                 "anger", "disgust", "fear", "sadness", "negative", "positive")

speech.df=tbl_df(sentence.list) %>%
  filter(school == "capitalism", word.count < 20) %>%
  select(sentences, anger:positive)
speech.df=as.data.frame(speech.df)
as.character(speech.df$sentences[apply(speech.df[,-1], 2, which.max)])

 [1] "This complaint, however, of the scarcity of money, is not always confined to improvident spendthrifts."                                        
 [2] "This money, however, was for a long time, received at the exchequer, by weight, and not by tale."                                              
 [3] "Bankruptcy is, perhaps, the greatest and most humiliating calamity which can befal an innocent man."                                           
 [4] "Poverty, though it no doubt discourages, does not always prevent, marriage."                                                                   
 [5] "The cheapness and plenty of good land encourage improvement, and enable the proprietor to pay those high wages."                               
 [6] "below its standard weight, the bank would, in this case, have lost only one percent."                                                          
 [7] "Its nominal price was a good deal lower than at present."                                                                                      
 [8] "The cheapness and plenty of good land encourage improvement, and enable the proprietor to pay those high wages."                               
 [9] "Not only ignorance and misinformation, but friendship, party animosity, and private resentment, are said frequently to mislead such assessors."
[10] "The cheapness and plenty of good land encourage improvement, and enable the proprietor to pay those high wages."                               

Communism:

speech.df=tbl_df(sentence.list) %>%
  filter(school == "communism", word.count < 20) %>%
  select(sentences, anger:positive)
speech.df=as.data.frame(speech.df)
as.character(speech.df$sentences[apply(speech.df[,-1], 2, which.max)])

 [1] "In actual history it is notorious that conquest, enslavement, robbery, murder, briefly force, play the great part."        
 [2] "The new nobility was the child of its time, for which money was the power of all powers."                                  
 [3] "Or perhaps Bastiat means, that a mode of production based on slavery is based on a system of plunder."                     
 [4] "In actual history it is notorious that conquest, enslavement, robbery, murder, briefly force, play the great part."        
 [5] "the rich grow rapidly richer, whilst there is no perceptible advance in the comfort enjoyed by the industrial classes."    
 [6] "One old woman was burnt to death in the flames of the hut, which she refused to leave."                                    
 [7] "Then one use value is just as good as another, provided only it be present in sufficient quantity."                        
 [8] "Dr. Simon, medical officer to the Privy Council, chose for this work the above mentioned Dr. Smith."                       
 [9] "In actual history it is notorious that conquest, enslavement, robbery, murder, briefly force, play the great part."        
[10] "In order to be fully prepared for his task, the working class revolutionary must also become a professional revolutionary."

Clustering of emotions

Here are the heatmaps of correlation matrix about sentiment respectively in capitalism and communism. Please note that the red regions means a strong positive correlation(close to but less or equal to 1) of two sentiment, blue regions means nearly no correlation(close to but more or equal to 0), and a white region shows some correlation.

Capitalism:

heatmap.2(cor(sentence.list%>%filter(school=="capitalism")%>%select(anger:positive)), 
          scale = "none", 
          col = bluered(100), , margin=c(6, 6), key=F,
          trace = "none", density.info = "none")
par(mar=c(4, 6, 2, 1))

Communism:

heatmap.2(cor(sentence.list%>%filter(school=="communism")%>%select(anger:positive)), 
          scale = "none", 
          col = bluered(100), , margin=c(6, 6), key=F,
          trace = "none", density.info = "none")
par(mar=c(4, 6, 2, 1))

We see that for both school, their heatmap are very similar. It makes sense that most positive sentiments are uncorrelated with most negative sentiment. But it is hard to tell any significant difference between two schools. So, let’s go with some more intuitionistic methods.

Bar plot and Pie chart of emotions

Here are the bar plot of mean score of each sentiment, and the pie chart of percentage of each sentiment. In the bar plot, all positive-related sentiments are in orange, and all negative-related sentiments are in their appropriate color.

sentence.list.capitalism <- sentence.list %>% filter(school=="capitalism")
emo.means1=colMeans(select(sentence.list.capitalism, anger:positive)>0.01)
col.use1=c("red2", "darkgoldenrod1", "chartreuse3","blueviolet", "darkgoldenrod1", "dodgerblue3", "darkgoldenrod1", "darkgoldenrod1", "black", "darkgoldenrod1")
barplot(emo.means1[order(emo.means1)], las=2, col=col.use1[order(emo.means1)], horiz=T,
        cex.names=0.7, main="Capitalism")

sentiment.df1 <- data.frame(
  group=c("anger", "anticipation", "disgust","fear", "joy", "sadness", "surprise", "trust", "negative", "positive"),
  value=emo.means1
)

sentiment.df1 <- sentiment.df1 %>% 
  mutate(perc = value / sum(value)) %>% 
  arrange(perc) %>%
  mutate(labels = scales::percent(perc))


ggplot(sentiment.df1, aes(x="", y=perc, fill=group)) +
  geom_bar(stat="identity", width=1, color="white") +
  coord_polar("y", start=0) +
  geom_text(aes(label = labels), 
            position = position_stack(vjust = 0.5),
            show.legend = FALSE,
            color = "white", size=3) +
  scale_fill_brewer(palette="Spectral") +
  labs(
    title = "Percentage of the sentiment in Capitalism",
    x = "",
    y = "",
  )

sentence.list.communism <- sentence.list %>% filter(school=="communism")
emo.means2=colMeans(select(sentence.list.communism, anger:positive)>0.01)
col.use2=c("red2", "darkgoldenrod1", "chartreuse3","blueviolet", "darkgoldenrod1", "dodgerblue3", "darkgoldenrod1", "darkgoldenrod1", "black", "darkgoldenrod1")
barplot(emo.means2[order(emo.means2)], las=2, col=col.use2[order(emo.means2)], horiz=T,
        cex.names=0.7, main="Communism")

sentiment.df2 <- data.frame(
  group=c("anger", "anticipation", "disgust","fear", "joy", "sadness", "surprise", "trust", "negative", "positive"),
  value=emo.means2
)

sentiment.df2 <- sentiment.df2 %>% 
  mutate(perc = value / sum(value)) %>% 
  arrange(perc) %>%
  mutate(labels = scales::percent(perc))


ggplot(sentiment.df2, aes(x="", y=perc, fill=group)) +
  geom_bar(stat="identity", width=1, color="white") +
  coord_polar("y", start=0) +
  geom_text(aes(label = labels), 
            position = position_stack(vjust = 0.5),
            show.legend = FALSE,
            color = "white", size=3) +
  scale_fill_brewer(palette="Spectral") +
  labs(
    title = "Percentage of the sentiment in Communism",
    x = "",
    y = "",
  )

These plots still looks similar, but now we can find some interesting pattern. In capitalism, joy is much higher than fear; but joy is lower than fear in communism. It looks like sentences in communism are more likely to express fear than in capitalism. However, is this the truth?

Let’s compare the mean sentiment score of capitalism and communism by combining the two bar plot. Here we find a really interesting point—— fear score is actually slightly higher in capitalism than in communism. So what happened? The truth is that sentences in capitalism are mostly more emotional than sentences in communism. The only exception is disgust.

sentiment.df.all <- data.frame(
  group=c("anger", "anticipation", "disgust","fear", "joy", "sadness", "surprise", "trust", "negative", "positive"),
  capitalism=emo.means1,
  communism=emo.means2
)

sentiment.df.all.long <- sentiment.df.all %>% pivot_longer(c("capitalism", "communism"))

g3 <- sentiment.df.all.long %>% ggplot(aes(x = group, y = value, fill = name)) +
  geom_col(position = "dodge") +
  labs(
    x = "",
    y = "mean score of sentiment",
    title = "Sentiment comparison: Capitalism vs. Communism"
  )

g3

Just for your curious, I made a bar plot of mean sentiment score for each authors in capitalism and communism. It is really funny to see that Marx is such an impassive person.

That’s all of the philosophy data story. I hope you have fun reading it!